Hi all,
I have a bit of an issue with determining why ListFTP doesn't seem to be
working or if it is why it's not forwarding attribute data and I'm sure the fix
is obvious however...
Requirement:
To list files from an FTP server and use the list to fetch the files. First
step is to report the attributes to the nifi log to check the code is working
before adding a fetchFTP processor.
Issue:
I have a listFTP processor that is running but does not appear to be
collecting attribute data which is to be passed to the nifi log to prove it's
working. Purposefully using incorrect login details registers an error as
expected suggesting login details are ok and the processor is running
suggesting remote-path/sub-directory-path/filename syntaxes maybe at issue.
Notes:
When building this on NiFi 1.9.2 it did seem to work for a while because the
provenance data was being collected for a short while but then NiFi hit on a
issue preventing it from emptying the 0.prov file automatically (I adjusted the
nifi config to make it use copies of these repository directories on the D:\
drive instead to by-pass what I think is a permissions issue). In version
1.11.4 I made this update before starting NiFi. The same problem regrading
attribute collection was present in 1.9.2 and 1.11.4 .
Any idea what I should be looking for? Do you have any suggestions about
directory path and filename syntax? Any links to relevant parts of the
documentation or message boards - I think I've looked at most of the obvious
ones- that maybe of use?
[details below]
Many thanks,
Guy
*************** Setup ********
NiFi 1.11.4, Windows Server 2016. FTP login via Filezilla or windows explorer
drops me at a root folder in which the target directory "fromfolder" is listed.
The target subdirectories use "##-##-##" as a name format e.g.,
"./fromfolder/20-04-22" and the files of interest are zip files with the name
convention of 11 digits e.g., "12345678901.zip". There are no special
permissions on the ftp site and ftp is use not sftp.
********** Processors used ******
listFTP - should return list file attributes for each target file
found
Hostname = value set to a variable of value "##.##.###.###"
User = value set to a variable of value "myusername"
Password = manually added string "secrettext"
Search Recursively = True
RemotePath = value set via a variable of value "./fromfolder"
or "fromfolder"
Path Filter Regex = "\d\d-\d\d-\d\d" ( target folders uses ##-##-##
name format )
File Filter Regex = "\d(11}\.zip ( target files use
###########.zip as a name format )
Other parameters = default values
Processor shows 5 tasks / 9.373 seconds
***
LogAttributes - should write all file attributes to the nifi-log
Parameters = default values
Terminate on success
Processor shows 0 tasks / 0 seconds
***
LogMessage - should write only the 'filename' attribute with a prefix in
the message of "ftplist"
Log prefix = "ftplist"
Log message = ${filename}
Other properties = default values
Terminate on success
Processor shows 0 tasks / 0 seconds
P Think Green - don't print this email unless you really need to
************************************************************************
The information contained in this e-mail and any files transmitted with it is
confidential and is for the exclusive use of the intended recipient. If you are
not the intended recipient please note that any distribution, copying or use of
this communication or the information in it is prohibited.
Whilst CAB International trading as CABI takes steps to prevent the
transmission of viruses via e-mail, we cannot guarantee that any e-mail or
attachment is free from computer viruses and you are strongly advised to
undertake your own anti-virus precautions.
If you have received this communication in error, please notify us by e-mail at
[email protected] or by telephone on +44 (0)1491 832111 and then delete the e-mail
and any copies of it.
CABI is an International Organization recognised by the UK Government under
Statutory Instrument 1982 No. 1071...
**************************************************************************
