Jim, There are a couple tickets related to the ListFile processors, I'm not sure if they apply to your situation but you might want to check them out to see if they help clarify anything.
- NIFI-1671 <https://issues.apache.org/jira/browse/NIFI-1671> relates to tests failing possible due to Windows account configurations, - NIFI-2383 <https://issues.apache.org/jira/browse/NIFI-2383> mentions files not being picked up, and - NIFI-3332 <https://issues.apache.org/jira/browse/NIFI-3332> relates to a race condition in the ListXXX processor's high watermark implementation. I agree with James Wing that your problem could be permissions, especially if your users send similar volumes of files. But if the missed files are sent in larger batches and/or sent over slower connections it might be the timestamp issue. Does your configuration move or delete files from the source directory after they are processed? If you remove processed files, you should be able to force the processor to pickup everything in the directory by clearing the processor state. If you do not remove processed files but can identify which ones weren't processed, you could update the file timestamps, with "touch" for example, to cause the processor to pick them up. Regards, Joe S On Tue, May 2, 2017 at 9:00 AM, James Keeney <[email protected]> wrote: > I have setup a processor chain with ListFile, FetchFile and then PutSFTP. > > Users FTP to the server that has NIFI and then NIFI is used to transfer > the files to our production servers using PutSFTP. > > When one user performs the transfer everything works, but when another > does it some of the files are not transferred. > > I have checked and all of the files made it onto the server with the > initial FTP transfer. I setup the PutSFTP so a reject or failure would > route the file to a directory using putfile. No files end up in that > directory so PutSFTP appears to be working correctly. So, It appears that > the breakdown occurs in the ListFile, FetchFile process. How would I go > about debugging this? No errors are thrown that I can see. > > Jim K. > -- > Jim Keeney > President, FitterWeb > E: [email protected] > M: 703-568-5887 <(703)%20568-5887> > > *FitterWeb Consulting* > *Are you lean and agile enough? * >
