Hi,

We are using Apache NiFi 1.3.0

We have a process flow to copy files from NFS to HDFS (with processors 
ListFile, FetchFile and PutHDFS)

In the NiFi process flow, ListFile is configured to listen to a directory on 
NFS. When a file (ex: x.csv) is being copied from a windows machine to NFS 
(while transfer is in the middle), a part file(x.csv.part) is created at NFS 
until transfer is complete.

ListFile has picked up this x.csv.part file and fetchFile picked up this to 
transfer to HDFS, didn’t update the file name back to x.csv in HDFS when 
transfer is complete.

But, in case a file from linux file system, while file copy to NFS is in 
progress it created (.x.csv) and when transfer is complete, at both NFS and 
HDFS, filename is updated to x.csv (from .x.csv).

Any thought how we can configure ListFile not to pickup these part files or any 
configurations in NiFi that fixes file names for these windows part files?

Appreciate your help.

Thanks,
Ravi Papisetti

Reply via email to